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Introduction 


The right to acquire a qualified higher education that can 
enhance chance for employment or realize carrier hopes requires 
being intellectually elite proven by strong competitive effort in 
college and university entrance exams because of the limited 
quota of the higher education programs (IHEP, 1998; Porter, 2002). 
Colleges and universities all around the world use a variety of clas- 
sification indicators of pre-collegiate academic success to decide 
which applicants should be accepted to university-level programs 
(OECD, 2007; IAU, 2009). Standardised admission test scores, such 
as the SAT, ACT, Math and SAT Verbal scores, measures high school 
academic performance (e.g., high school grade point average’GPA’ 
or percentile rank), matriculation or exit exam scores, student es- 
says, letters of recommendations, initial letters, formal interviews, 
foreign language proficiency scores for instance TOEFL, IELTS scores, 
and subject-specific test scores (e.g., Advanced Placement ‘AP’ 
tests in chemistry, biology, languages, etc.) have been used alone 
or combined with each other to create a selection index in which 
higher scores relate to higher levels of academic preparation and 
a greater likelihood of admittance (Culpepper & Davenport, 2009; 
Hambleton et al., 2009; Jang, & Roussos, 2007; Koys, 2010; Liang & 
Yuan, 2008; Luisa & Canado, 2010; Sohn & Ju, 2010; Zwick & Green, 
2007). In Turkey, the combination of the Student Selection Exami- 
nation (SSE) (which corresponds to a standardised test score) and 
the Secondary Education Weighted Achievement Point (SEWAP) 
(which corresponds to GPA scores) have been used to decide which 
students are eligible for certain high school programs through a 
system called the Student Selection and Placement System (SSPS) 
(SSPC, 2009a). Indeed, these and indicators reflect the degree of 
student achievement described in national educational curricular 
documents. 
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Abstract. This study presents findings col- 
lected from an analysis of Turkish Biology 
Curriculum Guidelines and of their align- 
ment with the curricular outcomes and 
the Student Selection Examination (SSE) 
for higher education. The analysis was 
performed in multiple dimensions: content 
areas, learning outcomes in terms of 
scientific research and process skills (SRPS), 
science-technology-society-environment 
(STSE), communication competencies, atti- 
tudes and values (CCAV) and cognitive ob- 
jectives. Webb's (2007) alignment criteria 
were used to investigate the relationship 
between curriculum standards and assess- 
ment frameworks. In this process, the biol- 
ogy questions asked in the 2010 SSE were 
examined in detail by 10 biology teachers 
who have been teaching biology courses 
at secondary schools. Although alignment 
consistency was high according to the 
depth-of-knowledge (DOK) consistency 
and categorical concurrence criteria, the 
SSE questions and curriculum outcomes 
were not fully aligned considering the low 
consistency of range of knowledge and 
balance of representation criteria. 
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During the last decade, the realisation of the precollege education reform movement has occupied 
the top place in Turkey's educational agenda, including the development of secondary school biology 
education curricula and associated instructional materials (MONE, 2009). The efforts for nationwide expan- 
sion and implementation were contemplated after a small-scale piloting of new instructional resources 
and are currently underway. To date, new national textbooks and other curricular materials have been 
developed for the elementary grades (between the age of 8 and 15). The development of textbooks and 
associated materials for the high school level is in the last phase of planning and/or development. Along 
with the revision of the curricula, the structure of central examinations are also expected to be modified 
to align with the assessment frameworks defined in the curriculum, which describe what students are 
expected to know or be able to do (Liu & Fulmer, 2008; Nasstrom & Henriksson, 2008; Sahin-Pekmez & 
Taskin-Can, 2007; Topcu & Sahin-Pekmez, 2009; Vos & Bos, 2005). The aim of this study was to investigate 
the relationship between assessment frameworks and curriculum standards. 

After nearly every curricular reform movement, a large number of research studies have examined 
the alignment of curriculum and assessment policies (e.g., Anderson, 2002; Brown, & Conley, 2007; Fulmer, 
2010; Liu & Fulmer, 2008; McDonald & Van Der Horst, 2007; Osta, 2007; Saderholm & Tretter, 2008; Sireci 
et al., 2009; Yan & Erduran, 2009). Among these studies, research on the curricular alignment of large- 
scale examinations comprises a much smaller body of literature (e.g., D'Agostino & Bonner, 2009; Liang 
& Yuan, 2008; Liu, & Fulmer, 2008; Vos & Bos, 2005). Because the context of these studies is specific to the 
country under discussion, Turkey's educational, social, and cultural variability necessitates research that 
regards the educational reflections of this diversity. Thus, these peculiar features make central educational 
assessment procedures not only indispensible but also influential and deeply affective. 

Secondary education in Turkey is somewhat distinctive. There are two main categories of high 
schools. General high schools aim to prepare students for university education, while vocational and 
technical high schools aim to equip students with job skills and place a lower emphasis on general 
education than do general high schools. Even within some types of schools, grade 10 students choose 
to enrol in different streams, which each place more emphasis on one subject than on another (e.g., 
science versus literature) (MONE, 2009). However school types differ, the Ministry of National Education 
(MoNE) determines the curriculum and sets standards for all elementary and secondary schools part 
of the centralised education system. The MONE is responsible for all educational services including 
policy decisions, curricula, the approval of textbooks and other instructional materials, governance 
and inspection of schools, hiring teachers, paying salaries and the maintenance of elementary and 
secondary schools. Student assessment is largely influenced by the curriculum guidelines produced 
by the Presidency of Teaching and Training Board (PTTB), and the textbooks used are published by the 
MoNE. The MoNE controls what is taught in schools by specifying the curriculum and approving the 
textbooks that can be used in the classrooms. As a consequence, student populations, levels of interest 
and attitudes toward science, and the rigours of science requirements and expectations are remarkably 
varied in the different types of high schools (OECD, 2007; Yildirim, 2004). 

Therefore, the unique cultural and educational contexts in Turkey, coupled with the enormous 
weight placed on large-scale examinations in relation to the development of new curricular materials for 
high school students, warrant an investigation of the alignment between large-scale examinations and 
the student learning at school. However, large scale examinations have examined only in unpublished, 
small-scale, Master’s-level theses in Turkey (Azar, 2010; Cetinkaya, 2009; Guzel, 2006; Coban, Aktas & 
Sulun, 2006; Kadayifci, 2007; Sesli, 2007; Uygun, 2008). 

In general, this study aims to analyse the content areas of the specified behaviours of curriculum 
for the 9""-12" grade (between the age of 16 and 19) and to investigate alignments of the SSE questions 
with the content and standards of the curriculum. For the purposes of this study, biology questions 
posed in the 2010 SSE booklets were analysed with respect to the standards of 2008 biology teaching 
program for 9-12" graders, which covered cognitive objectives and SRPS, STSE, and CCAV behaviours. 
The questions that guided the study were: (a) what are the specified outcomes of the biology curriculum 
as cognitive objectives and SRPS, STSE, CCAV behaviours with respect to the content areas? (b) What is 
the distribution of the SSE biology questions according to the chapters and learning areas of the curricu- 
lum? (c) What are the in-service biology teachers’ judgments about the alignment between curriculum 
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standards and the biology questions of SSE according to the Webb's (2007) alignment criteria? 
The 2008 Secondary Education 9-12" Grade Biology Lesson Teaching Program 


The vision of the program is to create biology literate students with the assumption that every- 
one can be successful at biology and enjoy learning. The program is composed of units that comprise 
the objectives directed to concept, principle and theory as well as objectives directed to competence, 
comprehension, attitude and value. The fundamental concepts in units related to ‘Cell, Organism and 
Metabolism; ‘Biological Diversity, Genetics and Evolution, and‘Environment and Humans’in 9" through 
12" grades are reflective of the subject content moving in parallel with learning concepts. All of the 
cognitive objectives are written with accompanying SRPS, STSE or CCAV behaviours in the biology 
teaching program. For example, the outcome of an assignment for 10" graders is described in the 
program as follows: ‘Students show the basic parts of a seed-bearing plant on a diagram (STSE 15; SRPS 
26)’ (PTTB, 2008). 


Student Selection Examination for Higher Education 


Every year approximately 1.5 million applicants (3% of the total population) take the entrance 
examinations to identify whether they are eligible for college or university programs in Turkey. The 
first stage of the SSE (SSE-1) is defined as ‘tests related to common subjects’ (SSPC, 2009b). In Turkey, 
beginning from the second year of high school (10° grade), students select fields of study rendered as 
science-mathematics, Turkish-mathematics, and Turkish-social. In the 9"" grade, all of the students are 
given identical types of courses to the above-mentioned as common subjects. With respect to these 
subjects, SSE-1 consists of four different tests: Turkish (Tur), Social Sciences (Soc-1), Mathematics (Math-1) 
and Science (Sci-1). All of the candidates were expected to answer SSE-1 questions. 

Unlike in the first stage, there are‘tests related to field subjects’in the second stage of the SSE (SSE-2) 
that cover the whole secondary school curricula (SSPC, 2009b). SSE-2 is made up of four different tests: 
Literature-Social Sciences (Lit-Soc), Social Sciences-2 (Soc-2), Mathematics-2 (Math-2) and Science-2 
(Sci-2). Candidates who would answer SSE-2 questions chose which tests were appropriate for them 
according to their school type, fields of study and the higher education programs to which they wanted 
to apply. All of the questions in the exam booklet were multiple choice questions that presented five 
alternatives. The total time allowed to complete the test is three hours and fifteen minutes for all. 


Methodology of Research 
Design of Research 


A document analysis method was used to analyse the process of the behaviours in the Biology 
Teaching Program (PTTB, 2008). To address the first research question of this study, the outcomes of 
the program were examined in detail regarding content areas. Numbers of listed outcomes under the 
standards of each chapter were determined to calculate the total numbers and percentages of cognitive 
objectives and SRPS, STSE, CCAV behaviours. Correspondingly, the SSE biology questions were examined 
in order to identify the alignment with the curriculum standards. 


Sample of Research 


Biology questions were reviewed by in-service biology teachers to investigate the conformity 
between standards and questions. The reviews were conducted with 10 biology teachers (60% female) 
working at secondary schools in the central part of the Trabzon province in Turkey. All of the reviewers 
were university graduates with a major degree in biological science, and most were experienced biology 
teachers with around 10 years of teaching experience. 
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Procedure 


The reviewers were provided with multiple choice SSE biology questions and biology curriculum 
objectives and recommended to complete the following four steps: relate class, unit, standard and 
objective with each question; identify the degree of convenience as compatible, partially compatible 
and weakly compatible between each biology question and the related objective along with the indi- 
cated STSE, SRSP, and CCAV if they are provided; explain what sort of content, skills and objectives are 
necessary to solve the biology questions; identify the relationship between the biology question and 
one of the related objective(s) under the standard. The reviewers followed the steps one by one for each 
of the SSE biology questions. The reviews lasted between 45-60 minutes. 


Data Analysis 


The collected data were analysed according to the criteria of the accepted alignment analysis 
model. Explanation of the model and the complete procedure of the study to fulfil the requirements 
for each criterion are summarised below. 

1. Categorical Concurrence Criterion: This criterion provides general information about whether 
the items included in the assessment measure the content regarding each standard. Normally, items 
placed in a test are expected to cover all of the standards. The SSE questions were analysed to deter- 
mine whether the items measured the content from each standard. In this process, hits from each re- 
viewer (provided by directly linking each biology question with its objectives) were coded to represent 
whether a match was found between the question and the objective(s) of the new biology program. 
The acceptable agreement coefficient was identified at least 0.63 by using a procedure developed by 
Subkoviak (1988). 

2. Depth-of-Knowledge (DOK) Consistency Criterion: DOK consistency criterion enables re- 
searchers to indicate cognitively what is elicited from students during the assessment processes and 
what they are expected to know or do as stated in the standards. The meaning of DOK consistency for 
the biology curriculum represents the degree of compatibility between the SSE questions and cognitive 
objectives and SRPS, STSE, CCAV behaviours listed under the standards (Webb, 2007). The reviewers 
compared all of the objectives under the standards with the SSE questions according to the cognitive 
process dimensions. The minimal acceptable level adopted as 50% among the questions coded to the 
standard below, at, or above the DOK level (Webb, 2007). 

3. Range of Knowledge Correspondence Criterion: This criterion mainly allows a concentrated 
comparison between appropriate breadth of coverage for a standard and the knowledge span of the 
standard(s) that students are required to answer correctly for the assessment items. The corresponding 
assessment items are expected to have at least 50% knowledge of correspondence for a minimal accept- 
able breadth of coverage (Webb, 2007). Therefore, ‘full compliance’ represents 50% or above, ‘partially 
compliance’ represents from 40% to 49% and ‘incompliance’ represents less than 40%. Judgments of the 
reviewers were examined concerning the depth and range of knowledge given in the objectives and 
whether these were adequate to the depth of knowledge required for students to answer the questions 
correctly. Moreover, the reviewers’ interpretations about what is required to solve the questions were 
analysed to enrich the quantitative data with qualitative findings in detail. 

4. Balance of Representation Criterion: Although the first three criteria enable us to judge the 
alignments of curriculum standards and assessments from many dimensions, they do not have any 
implication on how the questions are distributed among the objectives. The balance of representation 
criterion is used to indicate the degree to which an objective is given more emphasis on the assessment 
than another. An index is used to evaluate the distribution of items for each standard (Webb, 20007). To 
fulfil this criterion, the reviewers’ comparisons were coded for the index to decide which SSE question 
belonged to which objective(s) under the standard for all chapters of the new biology curriculum. In 
this index, ‘inacceptable’ indicates an index value below 0.60; ‘partially acceptable’ indicated an index 
value of 0.60 to 0.69 and ‘acceptable’ indicated an index value more than 0.70. Balance index formula 
and meanings of the symbols placed in the formula identified as, 
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Balance Index (BI) = 1—() | 1 /(O)—I(k)/ (HA) |) /2 
k=1 
Where O = Total number of objectives hit for the standard, 


| (k) = Number of items hit corresponding to objective (k), 
H = Total number of items hit for the standard. 


Results of Research 

In the first part of the study, the objectives listed below the standards belonging to new Biology Cur- 
riculum Programs were categorised as cognitive objectives and SRPS, STSE, CCAV behaviours and given 
below. In the second part of the study, data obtained from the alignment studies are given in detail. 


Findings Related to the Biology Curriculum Programs (9-12) 


At the beginning, the researchers analysed the curriculum regarding outcomes (cognitive, SRPS, 
STSE and CCAV) chapter based for each grade (9"°-12") from the curriculum guidelines. 


Table 1. Distribution of the objective in biology curriculum. 


Grades-Chapters Cognitive SRPS STSE CCAV 
9.1. Cell, organism, and metabolism 8 i 11 
9.2. Classification of living things and biological diversity 16 fi 10 
9.3. Conscious person — Livable environment r 6 12 11 
9th grade total 31 10 23 11 
10.1. Energy conversion in living things 14 4 6 
10.2. Cell division and reproduction 12 3 14 
10.3. Ecology of ecosystem 8 3 10 
10th grade total 34 5 23 
11.1. Plant biology Zt 4 3 
11.2. Genetic, genetic engineering, and biotechnology 20 4 19 
11.3. Community and population ecology 10 7 10 
11th grade total of 9 20 
12.1. Animal biology and human 47 4 5 
12.2. Origin of life and evolution 5 ° 11 
12.3. Protection of environment and rehabilitation 8 1 i 
12th grade total 60 8 17 


Findings from the percentage distribution of the cognitive objectives and SRPS, STSE, CCAV be- 
haviours in the curriculum guidelines were showed that CCAV behaviours remain limited only in one 
chapter at the 9°" grade level. The cognitive objectives mainly dominate the entire curriculum in an 
increasing manner from 9" grade through 12" grade. SRPS and STSE behaviours cover the rest of the 
curriculum objectives. Although the percentages of STSE behaviours are higher than the percentages 
of SRPS behaviours, both percentages decrease from 9" grade through 12" grade with a fluctuation at 
the 10" grade level. 
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Analysis of SSE Biology Questions 
Findings regarding the analysis of 2010 SSE biology questions are given in this section. There were 
10 questions asked in each of the Sci-1 and Sci-2 sections (see Appendix). The dispersion of these ques- 


tions according to the chapters and learning areas are given in the following table. 


Table 2. | The dispersion of the 2010 SEE questions according to learning areas and chapters. 


Sci-1 Sci-2 
Learning Areas Grades-Chapters 
N % N % 
Cell, organism, and 9.1. Cell, organism, and metabolism 3 30 - - 
metayeen 10.1. Energy conversion in living things - - 1 10 
11.1. Plant biology 2 20 1 10 
12.1. Animal biology and human - - 3 30 
Biologic Diversity, Genetic, 9.2. Classification of living things and biological diversity 2 20 - - 
aac voluon 10.2. Cell division and reproduction 1 10 - - 
11.2. Genetic, genetic engineering, and biotechnology 1 10 2 20 
12.2. Origin of life and evolution - - 1 10 
Environment and Human _‘ 9.3. Conscious person — Livable environment - - - - 
10.3. Ecology of ecosystem - - - - 
11.3. Community and population ecology 1 10 2 20 
12.3. Protection and rehabilitation of environment - - - - 
Total 10 100 10 100 


When the biology questions examined according to learning areas at the Biology curriculum pro- 
gram, half of the questions were asked from the ‘Cell, Organism, and Metabolism’ learning area, 35% of 
the questions were asked from the ‘Biological Diversity, Genetics, and Evolution’ learning area, and 15% 
of the questions were asked from the ‘Environment and Humans learning area. 

The Sci-1 test was expected to cover common subject areas — in other words, the 9" grade level. 
However, only half of the biology questions were asked at this level. The rest of the Sci-1 biology questions 
were taken from the 11" grade level (40%) and the 10" grade level (10%). The Sci-2 test was expected to 
cover field subject areas — in other words, the 10", 11" and 12" grade level. According to our examina- 
tion, 50% of the questions were taken from the 11" grade level, 10% of the questions were taken from 
the 10° grade level and 40% of the questions were taken from the 12" grade level. Finally, none of the 
questions were taken from the ‘Conscious person — liveable environment’ chapter at the 9" grade level 
that the Sci-1 test was expected to cover; none from the chapters of ‘Cell division and reproduction’ 
and ‘Ecology of ecosystem’ at the 10" grade level and none from the ‘Protection and rehabilitation of 
environment’ chapter at the 12" grade level that the Sci-2 test was expected to cover. 


Analysis of the 2010 SSE Biology Questions 
In this section, alignment studies that search for the relationship between the 2010 SSE biology 


questions and the objectives of the biology teaching program were performed by the 10 reviewers 
according to the four alignment criteria. 
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Criterion 1: Question Analysis for Categorical Concurrence 


Ten questions were asked about the ‘Cell, Organism, and Metabolism’ learning area. This learning 
area is constructed over four chapters for each grade 9" through 12". Each chapter has a different number 
of standards and objectives: the 9" grade ‘Cell, organism, and metabolism’ chapter has two standards 
and eight objectives; the 10" grade ‘Energy conversion in living things’ chapter has two standards and 
14 objectives; the 11" grade ‘Plant biology’ chapter has six standard and 27 objectives; and the 12" 
grade ‘Animal biology and human’ chapter has eight standard and 47 objectives. For this criterion, the 
reviewers firstly examined the questions and then specifically concentrated on the objectives of each 
standard that directly related to the SSE biology questions. 


Table 3. Categorical concurrence of the SSE biology questions and the standards. 


Questions Grade, Chapter, Standard, Objective Hits Concurrence 


Sci-1-21 9.1.1.2. Students define the organic and inorganic compounds that make the 0.9 concurrent 
structure of living things. 


Sci-1-22 9.1.1.2. Students define the organic and inorganic compounds that make the 0.9 concurrent 
structure of living things. 


Sci-1-23 9.1.1.4. Students explain structure of the cell and function of these structures 0.5 non-concurrent 
over cell model. 


Sci-1-24 11.1.2.1. Students explain the mechanisms that enable to take water from 07 concurrent 
ground and transport through the stem in plants (SRPS 26). 
11.1.2.2. Students explain how the transport of photosynthesis products occurs 


in plants. 
Sci-1-25 11.1.5.2. Students explain pollination over a flower figure (STSE 15; SRPS 26). 0.8 concurrent 
Sci-2-21 10.1.1.6. Students compare aerobic and anaerobic respiration. 0.7 concurrent 
Sci-2-22 11.1.5.3. Students explain fertilization in seed plants (STSE 15; SRPS 26). 0.8 concurrent 
Sci-2-24 12.1.6.7. Students explain arrival, transport, and response of the stimuli in sense 0.8 concurrent 


organs with examples (SRPS 7, 25; STSE 7, 15). 


Sci-2-25 12.1.6.11. Students explain the role of investigatory and regulatory system to 0.4 non-concurrent 
provide homeostasis with examples (SRPS 7). 


Sci-2-26 12.1.6.10. Students explain negative and positive feedback mechanism for 0.9 concurrent 
hormone secretion with examples (SRPS 7, 25; STSE 15). 


All reviewers provided hits for all of the questions as seen in Table 3. Standard 1 in the 9" grade 
related to three questions, standard 5 in the 11 grade related to three questions and standard 6 in 
the 12™ grade related to two questions hit by the reviewers. On the other hand, a question hit both 
standard 1 and standard 2 in the 10" grade. As a result, a total of six different standards appeared in 10 
biology questions among the 18 standards placed in the ‘Cell, Organism, and Metabolism’ learning area. 
One standard in 9" grade, four standards in 11" grade, and seven standards in 12" grade, totalling 12 
standards, did not take any hit. There was not an acceptable level of categorical concurrence not only 
between standard 1 at the 9 grade level and question sci-1-23 but also between standard 6 at the 12" 
grade level and sci-2-25. 

Detailed investigations revealed that the sci-2-25 slightly reflects the objectives sorted along with 
standard 2, which related to transporting gases, and standard 3, which related to the circulatory system, 
at the 12" grade level. For example, objective 4 sorted along with standard 2 required students to be 
able to explain the transportation of gases from alveolus to tissues or from tissues to alveolus (STSE 7, 15; 
SRPS 7, 25). Indeed, sci-2-25 examines whether students are able to explain the role of the investigatory 
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and regulatory system to provide homeostasis, not whether they could explanation the transportation 
procedures of respiratory gases in the body. From the results, it can be concluded that the content of 
the examined SSE questions mostly were met with a high agreement coefficient only for the related 
objectives sorted along with the standards, but because of the limited number of biology questions 
asked in SSE, the test remains far from covering all of the standards. 


Criterion 2: Question Analysis for DOK Consistency 


The objectives sorted with the standards for the ‘Cell, Organism, and Metabolism’ learning area 
were compared with the SSE questions on the basis of the complexity of knowledge required by each 
part to fulfil this criterion. At first, distribution of the SSE biology questions were realised according to 
their compatibility with the objectives along with standards, units and grade levels as cognitive objec- 
tives and SRPS, STSE, CCAV behaviours. Then the DOK consistency of the SSE biology questions for the 
biology teaching program objectives in terms of cognitive dimension was materialised. 


Table 4. Consistency of the SSE questions according to the curriculum outcomes. 


Chapter Objective Question Under At Above Consistency 
Cell, organism, and metabolism OAs: Sci-1-21 0.2 0.3 0.5 inconsistent 
Billed. Sci-1-22 0.5 0.5 inconsistent 
9.1.1.4. Sci-1-23 0.9 1.0 consistent 
Energy conversion in living things 10.1.1.6. Sci-2-21 0.1 0.7 0.2 consistent 
Plant biology 11.1.2.1. Sci-1-24 1.0 consistent 
1 Acou2, Sci-1-25 0.5 0.5 consistent 
11.1,533. Sci-2-22 0.2 0.6 0.2 consistent 
Animal biology and human 12.1.6.7. Sci-2-24 0.2 0.6 0.2 consistent 
12.1.6.11. Sci-2-25 0.3 0.2 0.5 inconsistent 
12.1610: Sci-2-26 0.2 0.8 consistent 


From Table 4, there are seven questions within the 10 questions belonging to the ‘Cell, Organism, 
and Metabolism’ learning area that are compatible with the corresponding CCAV, STSE, SRPS and Cog- 
nitive domains of the objectives along with the standards. From these findings, it could be said that, 
on the basis of the complexity of knowledge demanded cognitively by the objectives along with the 
standards, the SSE questions were close to each other. Therefore, DOK Consistency of the seven ques- 
tions was hit as ‘consistent’ 


Criterion 3: Question Analysis for Range of Knowledge Correspondence 


In this process, each reviewer examined the SSE biology questions and wrote down what types of 
knowledge or abilities students needed to know to solve these questions and what kind of objective(s) 
along with the standard can be given to the students. From the Table 5, it can be seen that within the 
10 questions belonging to the ‘Cell, Organism, and Metabolism’ learning area, only four questions had 
acceptable knowledge correspondence about related objectives along with the standards. Reviewer 
explanations about objectives and questions provided rich qualitative data in the area of knowledge 
correspondence. Moreover, reviewers evaluated the degree of compliance for each question to support 
their explanations. Acceptable knowledge correspondences were observed in only for four questions as 
a full compliance. From these findings, it could be said that a comparable span of knowledge expected 
of students by the objectives along with the standards for the teaching program is not the same or cor- 
respondent to the span of knowledge that students need to correctly answer the SSE questions. 
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Comparison of Knowledge Requirement and Objective 


Objective mainly focused on organic and inorganic compounds in the body 
of biological organism. Without understanding the anabolic and catabolic 
reaction mechanisms in other words chemical conversions of organic and 
inorganic compounds it is not possible to solve the problem. 


Solution of the problem requires comprehensive knowledge about 
enzymes. Question try to investigate enzyme concept, structure, and 
reaction mechanism understood with its results by students. Objective 
was remained too shallow for the question. 


Question investigates structures and functions of organelles over a 
sample that foresight gaining tree dimension, becoming ready for ac- 
tion, and transport to the area of usage in a cell stages of a polypeptide 
chain to secretion instead of a cell model. Although question constructed 
over an abstract sample that occurs in a cell, knowing protein synthesis 
mechanism is clearly facilitate the solution. 


Question was probing water transport in plants through radioactively 
signed hydrogen atom included to water sample and transport of photo- 
synthesis products. Given stress on photosynthesis limited with knowledge 
of reactants (water) and products (sucrose) of the reaction more than the 
knowledge on hydrogen atoms in structure of sucrose as a product of 
photosynthesis reaction derived from water's hydrogen atoms. 


Samples of reproduction adaptations in entomophilies listed at the answer 
choices by giving focus on pollination in flowering plants. There is a 
slight mention on adaptation concept in addition to pollination. Question 
neither requires any figure or model impression nor verbal or written 
summarization. 


Students was enabled to compare ways of energy production in living 
organisms over muscle cells that can make oxidative phosphorylation like 
other somatic cells in case of oxygen existence and lactic acid fermenta- 
tion in case of oxygen debt, and facultative anaerobe bacteria that can 
live in either the presence and absence of oxygen. 


Cases occurring in flowering plants randomly enumerated starting from 
arrival of pollen to the stigma until formation of endosperm tissue at the 
choices of this question. Students can easily find true answer if they 
achieve to recall basic knowledge about fertilization in flowering plants. 


Cognitive objective mainly focused on stimuli acceptance and transport 
procedures in sense organ. But, question investigates awareness of a 
specific feature of these organs. Given real life situations direct students 
to notice contribution to biology for understand life. 


The question was asked to investigate the regulatory system's supervision 
on respiratory system. Students expected to make comparisons among 
real life situations listed at the choices of this question. But given samples 
also required to know function and structure of circulatory, respiratory 
and neural system. 


Whether the students explain the regulation of hormone secretion through 
positive and negative feedback was aimed to investigate by asking this 
question. From this point of view question and cognitive objective seems 
to be convenient. But students also need to know factors that can affect 
blood pressure. 
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Criterion 4: Question Analysis for Balance of Representation 


To find how the questions asked were distributed among the objectives listed below the same 
standards, a balance index calculation for each question was utilised. 


Table 6. Balance Index of SSE Biology Questions. 


Chapter Question O H | Bl Balance Level 
Cell, organism, and metabolism Sci-1-21 3 5 [2,3,4= 9,1,1 0.4 unacceptable 
Sci-1-22 Z s [2,4= 10,1 0.4 unacceptable 
Sci-1-23 2 a) 11,4= 3,9 0.3 unacceptable 
Plant biology Sci-1-24 2 3 I1,2= 8,5 0.67 partially 
Sci-1-25 4 7 11,2,3,7= 1,6,1,2 0.79 acceptable 
Energy conversion in living things Sci-2-21 5 6 [2,3,5,6,8= 5,1,8,4,2 0.2 unacceptable 
Plant biology Sci-2-22 3 i I1,2,3= 1,3,7 0.71 acceptable 
Animal biology and human Sci-2-24 2 11 I6,7= 8,10 0.68 partially 
Sci-2-25 a 11 I2,3,4= 10,4,6 0.59 unacceptable 
Sci-2-26 4 11 18,9,10,11= 2,10,4,1 We) acceptable 


From Table 6, only three questions could be classified into the acceptable category for balance of repre- 
sentation criterion. This criterion shows that the SSE questions were not distributed equally within objectives 
listed below the standards. It means that some objectives were given more emphasis on the SSE questions 
than others. Therefore, many standards were ignored and acceptable balances were not established. 


Discussions and Implications 


According to the outcome analysis results of the biology teaching program, cognitive objectives are 
prominently emphasised. When the weight of outcomes was examined from the 9" grade level to 12 grade 
level, the ratio of cognitive objectives did increase from 40% to 70% (Table 1).In a teaching program, cognitive 
objectives provide a means to account for the subject knowledge that students acquire by the end of the 
learning process., Merely emphasising the cognitive objectives in this curriculum simply means that students 
have to continue to acquire the tremendous knowledge load As in the curricula that have previously failed 
(Ayas, Cepni & Akdeniz, 1993; Cepni & Cil, 2009) while all around the world education trends are rapidly 
changing in response to studies about induction of curriculum; active student participation opportunities; 
student-centred approaches in classrooms; the development of STSE, SPRS, and CCAV behaviours. 

Further, SSE questions were analysed by means of only biology questions that are prepared and con- 
ducted by the SSPC. Whereas questions related to the 9" grade level common subject field were supposed to 
be placed in the Sci-1 test, only 50% of the biology questions on the test represented this level. On the other 
hand, the test did not include biology questions related to the ‘Conscious person — Liveable environment’ 
chapter at the 9" grade level, which was regarded as part of the common subject field for all students. This 
disparity between preparation and test may account for the lowness of the average correct response to biol- 
ogy questions placed in the Sci-1 test. Expecting students who did not have biology courses after 9" grade 
level to solve questions related to the biology subject field and then deciding eligibility for higher education 
programs according to the scores obtained through these questions harms the validity of the measurement. 
Straying from the principles identified in the published examination guide by the SSPC also deeply affects 
the psychology of the candidates (Tansel & Bircan, 2006). 

Through the analysis of the balance between the questions and the new biology program, positive 
significant results were obtained regarding Categorical Concurrence Criterion. However, these results do 
not resemble Saderholm and Tretter (2008)’s study results for analysing similarities and differences between 
the assessment structures for physics and the Benchmarks for Science Literacy (Benchmarks), National Sci- 
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ence Education Standards (NSES), National Assessment Governing Board (NAEP) and Trends in International 
Mathematics and Science Study (TIMMS), which give direction to science curricula all around the world. 
Increasing the number of questions would barely enable the exam to cover more subjects at the same time, 
and also one question can cover more than one standard (Table 2). Therefore, SSPC needs to pay attention 
to preparing questions that relate to all program standards. 

The DOK consistency analyses were revealed the coherence between the mental complexities of the 
SSE exam and the cognitive level of the related objective because of understand and knowledge level cogni- 
tive requirements. Liu and Fulmer (2008) stated that tests in China and Singapore require a higher level of 
thinking ability according to the standards, while there is a level of compliance between the examination and 
standards cognitive level in the United States. In this context, Turkey is similar to the US with regard to the SSE 
examination and its compatibility with preparation programs. However, both outcomes and questions that 
include low cognitive competencies negatively affect the measurement. In PISA the required evaluation skills 
were classified as induction, deduction, logical thinking, system-based thinking and critical thinking, which 
are fundamental cognitive skills (Fensham, 2009; Hatzinikita, 2008; Nentwig et al., 2009). It can be seen asa 
positive point that the SSE exam contains very few knowledge-level questions. Consequently, questions that 
assess competence at the level of analysis, evaluation, and creative cognition should be given more space 
in the SSE examinations. 

The analyses of the Range of Knowledge Criterion correspondences were indicated that only four of 10 
questions could be answered with the knowledge gained through the outcomes of the analysed learning 
area. In many situations, questions required more in-depth information from the acquisition expressed in the 
program. In this respect, the depth of knowledge stated in the standards is insufficient for students to solve 
the questions. These results strengthen the inclination of students to the tutors and become alienated from 
the school atmosphere because they feel that schools are not able to help them crack the SSE and achieve 
their higher education goals. As Riffert (2005) suggests, examinations have to be adapted to schools rather 
than adapting schools according to examinations. 

Although the questions in the SSE were prepared to include standards, the balance of representation 
criterion was not mostly neglected among the outcomes of a particular standard. Only three of 10 examined 
questions investigated in this study covered a large number of outcomes. In other words, a large number of 
outcomes that students are expected to achieve are left out of the scope of assessment in the examination. 
It is believed that a coherency between curriculum and central examinations shows the importance of the 
curriculum and it shows how successfully students acquire the outcomes (Kjellstrom & Pettersson, 2005). 
Finally, having common characteristics between the SSE and the curriculum will contribute to the assessment 
and the improvement of the curriculum. 


Conclusion 


Superficial assessment results obtained through the analysis for the SSE questions during this study 
indicated that the outcomes of the teaching program had a consistency with the SSE questions. Based on these 
results, all decision and policy makers argue that the current SSE is valid and reliable and at the same time it 
reflects the nature of the current curriculum (Cepni & Cil, 2009). However, when the SSE were analysed using 
four alignment criteria, it was found that the questions is not fulfilling the range of knowledge and balance 
of representation criterions which are the measure for alignment between the curriculum and assessment. 
In this case, there is a strong need for the reassessment of all the SSE questions, including all subjects and 
tests, to reach for stronger conclusions. 

Finally, it could be concluded that validity and reliability of the examination for higher education eli- 
gibility needs to be improved regarding the findings and discussions. To meet this need, the policy makers 
should first analyse the contents and the nature of questions asked in the central examinations. Additionally, 
to become successful at preparing students for international examination, all educators and researchers 
should be aware of the fact that the new biology teaching program should include concepts of scientific 
literacy, nature of science, basic science concepts, the relationships between science, technology, society and 
environment and the reflections of fundamental behaviours of SRPS, STSE and CCAV in practice. If students 
grasp all these requirements, they will be very successful at the central examinations in addition to using 
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the knowledge in solving daily life problems. Theoretically, this might sound possible, but how all of these 
advanced ideas and attainments can be put into practice will remain the main problem that must be dealt 
with in the near future. 


References 


Anderson, L. W. (2002). Curricular alignment: A re-examination. Theory into Practice, 41 (4), 255-260. 

Ayas, A., Cepni, S., & Akdeniz, A. R. (1993). Development of the Turkish Secondary Science Curriculum. Science 
Education, 77(4), 433-440. 

Azar, A. (2010). The effect of critical thinking dispositions on students’achievementin selection and placement exam 
for university in Turkey. Journal of Turkish Science Education, 7 (1), 61-73. 

Cepni, S., & Cil, E. (2009). Introduction of the New Science and Technology Curriculum. Pegema Publications, Ankara. 

Brown, R.S., & Conley, D.T. (2007). Comparing state high school assessments to standards for success in entry-level 
university courses. Educational Assessment, 12(2), 137-160. 

Cetinkaya, S. (2009). The evaluation of Turkish questions asked in SSE with the aspect of taxonomy. Unpublished Master 
Dissertation, Gazi University, Ankara, Turkey. 

Coban, A., Aktas, M., & Sulu, A. (2006). An evaluation of biology course program in respect of SSE questions. Journal 
of Erzincan Education Faculty, 8 (1), 23-36. 

Culpepper, S. A., & Davenport, E. C. (2009). Assessing Differential prediction of college grades by race/ethnicity with 
a multilevel model. Journal of Educational Measurement, 46 (2), 220-242. 

D’Agostino, J. V., & Bonner, S. M. (2009). High school exit exam scores and university performance. Educational As- 
sessment, 14 (1), 25-37. 

Fensham, P. J. (2009). Real world contexts in PISA science: Implications for context-based science education. Journal 
of Research in Science Teaching, 46 (8), 884-896. 

Fulmer, G.W. (2010). Estimating critical values for strength of alignment among curriculum assessments, and instruction. 
Paper presented at the annual meeting of AERA, San Francisco, US. 

Guzel, M. (2006). A research on SSE and SPE Turkish questions. Unpublished Master Dissertation, Gazi University, 
Ankara, Turkey. 

Hatzinikita, V., Dimopoulos, K., & Christidou, V. (2008). PISA test items and school textbooks related to science: A 
textual comparison. Science Education, 92 (4), 664-687. 

Hambleton, R. K., Sireci, S.G., & Smith, Z. R. (2009). How Do Other Countries Measure Up to the Mathematics Achieve- 
ment Levels on the National Assessment of Educational Progress? Applied Measurement in Education, 22 (4), 376-393. 

IHEP, Institute for Higher Education Policy (1998). Reaping the benefits: defining the public and private value of going 
to college. Pricing, and Productivity. Washington DC, US. 

IAU, International Association of Universities (2009). World higher education database single user 2009. Palgrave 
Macmillan’s Global Academic Publishing, Hampshire, UK. 

Jang, E. E., & Roussos, L. (2007). An Investigation into the Dimensionality of TOEFL Using Conditional Covariance- 
Based Nonparametric Approach. Journal of Educational Measurement, 44 (1), 1-21. 

Kadayifci, K. G. (2007). Investigation of the compatibility of the chemistry questions asked in high school and university 
entrance exam with the program. Unpublished Doctoral Dissertation, Gazi University, Ankara, Turkey. 

Kjellstrom, K., & Pettersson, A. (2005). The curriculum’s view of knowledge transferred to national tests in mathematics 
in Sweden. International Journal on Mathematics Education ZDM, 37(4), 308-316. 

Koys, D. (2010). GMAT versus alternatives: Predictive validity evidence from central Europe and the Middle East. 
Journal of Education for Business, 85, 180-185. 

Liang, L. L., & Yuan, H. (2008). Examining the alignment of Chinese national physics curriculum guidelines and 12" 
grade exit examinations: A case study. International Journal of Science Education, 30(13), 1823-1835. 

Liu, O. L., & Wilson, M. (2009). Gender Differences in Large-Scale Math Assessments: PISA Trend 2000 and 2003. Ap- 
plied Measurement in Education, 22 (2), 164-184. 

Liu, X., & Fulmer, G. (2008). Alignment between the science curriculum and assessment in selected NY state regents 
exams. Journal of Science Education and Technology, 17 (4), 373-383. 

Luisa, M., & Canado, P. (2010). English language teaching in the European Higher Education Area: from policy to 
practice. Innovation in Language Learning and Teaching, 4 (1), 53-69. 

McDonald, R., &Van Der Horst, H. (2007). Curriculum alignment, globalization, and quality assurance in South African 
higher education. Journal of Curriculum Studies, 39 (1), 1-9. 

MONE (2009). National education statistics formal education 2008- 2009. Strategy Development Presidency Depart- 
ment of Publications, Ankara. 

Nasstrom, G., & Henriksson, W. (2008). Alignment of standards and assessment: A theoretical and empirical study of 
methods for alignment. Electronic Journal of Research in Educational Psychology, 6 (3), 667-690. 

Nentwig, P., Roennebeck, S., & Schoeps, K. (2009). Performance and levels of contextualization in a selection of OECD 


84 


Ws 


Journal of Baltic Science Education, Vol. 10, No. 2, 201 1 


ISSN 1648-3898 INVESTIGATION THE ALIGNMENT BETWEEN SCHOOL LEARNING AND ENTRANGE 
EXAMINATIONS THROUGH ITEM ANALYSIS 
(P. 73-86) 


countries in PISA 2006. Journal of Research in Science Teaching, 46 (8), 897-908. 

OECD (2007). Reviews of national policies for education: Basic education in Turkey. OECD Publishing. 

Osta, |. (2007). Developing and piloting a framework for studying the alignment of mathematics examinations with 
the curriculum: The case of Lebanon. Educational Research and Evaluation, 13 (2), 171-198. 

Porter, K. (2002). The value of college degree. ERIC Digest ERIC Clearinghouse on Higher Education, One DuPont 
Circle, Suite 630, Washington D.C., US. 

PTTB (2008). 9-12" grade biology course teaching program. MONE, Ankara, Turkey. 

Riffert, F. (2005). The use and misuse of standardized testing: A Whiteheadian point of view. Interchange, 36(1-2), 
231-252. 

Saderholm, J. C., & Tretter, T. R. (2008). Identification of the most critical content knowledge base for middle school 
science teachers. Journal of Science Teacher Education, 19(3), 269-283. 

Sahin-Pekmez, E., & Taskin-Can, B. (2007). The reflection of 2000 and 2004 science curricula on the prospective 
teachers. Journal of Turkish Science Education, 4 (2), 109-118. 

Sesli, A. (2007). A comparative analysis of the questions asked in exam by biology teachers and the questions asked 
in the university entrance exams according to Bloom's taxonomy. Unpublished Master Dissertation, Karadeniz Technical 
University, Trabzon, Turkey. 

Sireci, S. G., Hauger, J. B., Wells, C. S., Shea, C., & Zenisky, A. L. (2009). Evaluation of the Standard Setting on the 2005 
Grade 12 National Assessment of Educational Progress Mathematics Test. Applied Measurement in Education, 22 (4), 339- 
350. 


SSPC (2009a). Number of applicants and placements in higher education programs (Data file). Retrieved 03/02/2011, 
from Available from 


SSPC (2009b). 2009 Student selection and placement system guide. Meteksan Co. Ankara. 

Sohn, S. Y., & Ju, Y. H. (2010). Conjoint analysis for recruiting high quality students for college education. Expert 
Systems with Applications, 37, 3777-3783. 

Subkoviak, M. J. (1988). A practitioner’s guide to computation and interpretation of reliability indices for mastery 
tests. Journal of Educational Measurement, 25, 47-55. 

Tansel, A., & Bircan, F. (2006). Demand for education in Turkey: A Tobit analysis of private tutoring expenditures. 
Economics of Education Review, 25 (3), 303-313. 

Topcu, M. S., Sahin-Pekmez, E. (2009). Turkish middle school students’ difficulties in learning genetics concepts. 
Journal of Turkish Science Education, 6 (2), 55-62. 

Uygun, N. (2008). Validity of science items in the student selection test in Turkey. Unpublished Master Dissertation, 
Middle East Technical University, Ankara, Turkey. 

Vos, P., & Bos, K. (2005). The mathematics curriculum in the Netherlands: Measuring curricular alignment using 
TIMSS-99. Educational Research and Evaluation, 11 (2), 201-219. 

Webb, N. L. (2007). Issues related to judging the alignment of curriculum standards and assessments. Applied Mea- 
surement in Education, 20(1), 7-25. 

Yan, X., & Erduran, S. (2009). Arguing online: case studies of pre-service science teachers’ perceptions of online tools 
in supporting the learning of arguments. Journal of Turkish Science Education, 5 (3), 2-31. 

Yildirim, A. (2004). Student assessment in high-school social-studies courses in Turkey: teachers’ and students’ 
perceptions. International Review of Education, 50, 157-175. 

Zwick, R., & Green, J. G. (2007). New Perspectives on the Correlation of SAT Scores, High School Grades, and Socio- 
economic Factors. Journal of Educational Measurement, 44 (1), 23-45. 


Appendix 

Sample Biology Questions in 2010 Student Selection Examination 

2010 - SSE / Sci-1 

29. Which of the below characteristic do not seen in reptiles that is one of the vertebrate animal 


group? 
a) Internal fertilisation 


b) Dirty and clear blood mixed circulation 
c) Pulmonary respiration 
d 


) Egg laying 
) Constant body temperature 


M 
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24. When a hydrogen atoms’ marked water given plant grow up, marked hydrogen is encoun- 
tered at sucrose in fruit. 
Until this marked water molecules arrive to the fruit, passed through from structure 
I. xylem, 
ll. root hair, 
Il. phloem, 
IV. leave mesophyll, 
in which row? 


a) I-Il-IN-IV 
b) Il-I-IV-Ill 
c) W-Im-WV-| 
d) I-Il-l-IV 
e) Il-IW-I-| 


2010 - SSE / Sci-2 


22. In flowering plants, which of the below occasions occurs after fertilisation? 
a) Pollen germination 


b) Endosperm formation 
c) Pollen transfer to stigma 
d) Antipode cell formation 


) 
) Reach of pollen tube to embryo sac 


M 


27. \In a research, it is determined that each phloem cells taken from carrot plant root grow up 
to the whole plant in culture environment. 
According to this research, about the ob- 
tained whole plant, 
I. Cells lost some genes when they dif- 
ferentiated and specialised. 
ll. All the cells have same genes in their 


Root of carrot plant 


Transversal cross 
ection of root 







Phloem 





nucleus. i ae 
Ill. Plants are same with the original " O° 
plants soni 
= deapece ‘ a 
decisions of which is/are true? ee \¥ 
a) Only! Embryonic 1 
b) Only Il development | 
c) Onlylll 
d) | and Ill Cells in nutrition Seedling taken to 
e) lland Ill environment the culture 
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